Transferring Physical Skills From Humans to Robots: Multimodal Programming by Demonstration for In-Contact Tasks

A new generation of industrial quality robotic manipulators is currently being released on the international market. State of the art over-actuated manipulators are now starting to be flanked with a significant proprioceptive capacity (i.e. force/torque sensing at each joint). Joint proprioception can be further extended by force/torque sensing at the robot’s wrist and multi-fingered robotic hands provided with torque and tactile sensing. Therefore, although still fulfilling the key properties of traditional industrial robots in their motor actuation (i.e. accuracy, high performance, control, reliability and durability), a new class of commercial robots can now receive significant sensory feedback, enhancing their potential for adaptive dexterous manipulation. Against a relatively limited financial investment (starting from approximately 20.000 US dollars), manufacturers can now easily deploy this novel option in their workshops. The manufacturing process can integrate unprecedented possibilities in form of low cost, flexible, industrial quality manipulators under extended proprioceptive monitoring. However, in contemporary robotics, major hardware improvements require the balanced coupling with comparably efficient soft methods. Unfortunately, when trying to exploit the full potential of the new robots for a broad range of industrial applications, the current unbalance between hard and soft methods emerges. Nowadays, the deployment of true, skillful and flexible manipulation can only be achieved in front of significant costs due to the rather delicate and complex process of dedicated software development. Nevertheless, the current generation of robotic manipulators seems ready for major advancement in the field of HRI, in particular for the transfer of physical skills from humans to robots. Over the last decade, a significant body of work has focused on the generation of trajectories, reproducing at the level of the robot’s end effector the ones demonstrated by a human instructor, and on the system’s capacity to generalize these trajectories to novel situations (e.g. coping with obstacles or other constraints). This approach, namely programming by demonstration (hereafter PbD [1]), can be implemented as a general architecture for flexible use. In kinesthetic approaches, the end effector is physically grabbed by the human demonstrator, and moved in space to directly